Depth camera assembly using fringe interferometery via multiple wavelengths

ABSTRACT

A depth camera assembly (DCA) determines distances between the DCA and objects in a local area within a field of view of the DCA. The DCA projects a series of sinusoidal patterns each having different wavelengths into the local area DCA and captures images of the sinusoidal patterns via a sensor. Optical filters configured to pass different wavelengths of light are positioned within the sensor to form regions including adjacent pixels. Hence, pixels in a region capture light having a wavelength passed by an optical filter corresponding to the region. If the DCA projects sinusoidal patterns having different wavelengths at different times, the sensor is gated with an illumination source so regions of the sensor capturing light having a specific wavelength capture light while the illumination source emits the specific wavelength and not while the illumination source emits other wavelengths.

BACKGROUND

The present disclosure generally relates to virtual or augmented realitysystems and more specifically relates to headsets for virtual realitysystems that obtain depth information of a local area.

Providing virtual reality (VR) or augmented reality (AR) content tousers through a head mounted display (HMD) often relies on localizing auser's position in an arbitrary environment and determining a threedimensional mapping of the surroundings within the arbitraryenvironment. The user's surroundings within the arbitrary environmentmay then be represented in a virtual environment or the user'ssurroundings may be overlaid with additional content.

Conventional HMDs include one or more quantitative depth cameras todetermine surroundings of a user within the user's environment.Typically, conventional depth cameras use structured light or time offlight to determine the HMD's location within an environment. Structuredlight depth cameras use an active illumination source to project knownpatterns into the environment surrounding the HMD. However, structuredlight commonly requires a pattern that is projected to be configured sodifferent portions of the pattern include different characteristics thatare later identified. Having different characteristics of differentportions of the pattern causes significant portions of a resulting imageof the projected pattern to not be illuminated. This inefficiently usesa sensor capturing the resulting image; for example, projection of thepattern by a structured light depth camera results in less than 10% ofsensor pixels collecting light from the projected pattern, whilerequiring multiple sensor pixels to be illuminated to perform a singledepth measurement.

Time of flight depth cameras measure a round trip travel time of lightprojected into the environment surrounding a depth camera and returningto pixels on a sensor array. While time of flight depth cameras arecapable of measure depths of different objects in the environmentindependently via each sensor pixel, light incident on a sensor pixelmay be a combination of light received from multiple optical paths inthe environment surrounding the depth camera. Existing techniques toresolve the optical paths of light incident on a sensor pixel arecomputationally complex and do not fully disambiguate between opticalpaths in the environment.

SUMMARY

A headset in a virtual reality (VR) or augmented reality (AR) systemenvironment includes a depth camera assembly (DCA) configured todetermine distances between a head mounted display (HMD) and one or moreobjects in an area surrounding the HMD and within a field of view of animaging device included in the headset (i.e., a “local area”). The DCAincludes the imaging device, such as a camera, and an illuminationsource that is displaced by a specific distance relative to theillumination source. The illumination source is configured to emit aseries of periodic illumination patterns (e.g., a sinusoid) into thelocal area. Each periodic illumination pattern of the series is phaseshifted by a different amount. The periodicity of the illuminationpattern is a spatial periodicity observed on an object illuminated bythe illumination pattern, and the phase shifts are lateral spatial phaseshifts along the direction of periodicity. In various embodiments, theperiodicity of the illumination pattern is in a direction that isparallel to a displacement between the illumination source and a centerof the imaging device of the DCA.

The imaging device captures frames including the periodic illuminationpatterns via a sensor including multiple pixels and coupled to aprocessor. For each pixel of the sensor, the processor relatesintensities captured by a pixel in multiple images to a phase shift of aperiodic illumination pattern captured by the multiple images. From thephase shift of the periodic illumination pattern captured by the pixel,the processor determines a depth of a location within the local areafrom which the pixel captured the intensities of the periodicillumination pattern from the HMD. Each pixel of the sensor mayindependently determine a depth based on captured intensities of theperiodic illumination pattern, optimally using the pixels of the sensorof the DCA.

In various embodiments, the illumination source simultaneously emitsmultiple periodic illumination patterns having different wavelengths anddifferent phase shifts. The sensor includes a set of optical filterspositioned to form multiple regions of adjacent pixels. Differentoptical filters are configured to pass different wavelengths of light,causing different regions of the sensor to capture intensity informationfrom different wavelengths of light. The optical filters may havedifferent positions relative to each other in various embodiments,allowing different embodiments of the sensor to have regions of pixelscapturing intensity information about different wavelengths of light indifferent positions relative to each other.

Alternatively, the illumination source emits periodic intensity patternshaving different wavelengths at different times. In such aconfiguration, different pixels of the sensor capture intensityinformation from different wavelengths of light. Additionally, thesensor is gated with the illumination source so regions of the sensorcapturing intensity information from a wavelength of light captureintensity information when the illumination source emits a periodicintensity pattern having the wavelength of light but do not captureintensity information when the illumination source emits periodicintensity patterns having other wavelengths.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system environment for providing virtualreality or augmented reality content, in accordance with an embodiment.

FIG. 2 is a diagram of a head mounted display (HMD), in accordance withan embodiment.

FIG. 3 is a cross section of a front rigid body of a head mounteddisplay (HMD), in accordance with an embodiment.

FIG. 4 is an example of light emitted into a local area and captured bya depth camera assembly, in accordance with an embodiment.

FIG. 5 is an example of using multiple frequencies of a continuousintensity pattern of light emitted by a DCA to identify a phase shiftfor a pixel of the sensor, in accordance with an embodiment.

FIG. 6 is an example sensor included in an imaging device of a depthcamera assembly, in accordance with an embodiment.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles, or benefits touted, of the disclosure described herein.

DETAILED DESCRIPTION

System Overview

FIG. 1 is a block diagram of one embodiment of a system environment 100in which a console 110 operates. The system environment 100 shown inFIG. 1 may provide augmented reality (AR) or virtual reality (VR)content to users in various embodiments. Additionally or alternatively,the system environment 100 generates one or more virtual environmentsand presents a virtual environment with which a user may interact to theuser. The system environment 100 shown by FIG. 1 comprises a headmounted display (HMD) 105 and an input/output (I/O) interface 115 thatis coupled to a console 110. While FIG. 1 shows an example systemenvironment 100 including one HMD 105 and one I/O interface 115, inother embodiments any number of these components may be included in thesystem environment 100. For example, there may be multiple HMDs 105 eachhaving an associated I/O interface 115, with each HMD 105 and I/Ointerface 115 communicating with the console 110. In alternativeconfigurations, different and/or additional components may be includedin the system environment 100. Additionally, functionality described inconjunction with one or more of the components shown in FIG. 1 may bedistributed among the components in a different manner than described inconjunction with FIG. 1 in some embodiments. For example, some or all ofthe functionality of the console 110 is provided by the HMD 105.

The head mounted display (HMD) 105 presents content to a user comprisingaugmented views of a physical, real-world environment withcomputer-generated elements (e.g., two dimensional (2D) or threedimensional (3D) images, 2D or 3D video, sound, etc.) or presentscontent comprising a virtual environment. In some embodiments, thepresented content includes audio that is presented via an externaldevice (e.g., speakers and/or headphones) that receives audioinformation from the HMD 105, the console 110, or both, and presentsaudio data based on the audio information. An embodiment of the HMD 105is further described below in conjunction with FIGS. 2 and 3. The HMD105 may comprise one or more rigid bodies, which may be rigidly ornon-rigidly coupled to each other together. A rigid coupling betweenrigid bodies causes the coupled rigid bodies to act as a single rigidentity. In contrast, a non-rigid coupling between rigid bodies allowsthe rigid bodies to move relative to each other.

The HMD 105 includes a depth camera assembly (DCA) 120, an electronicdisplay 125, an optics block 130, one or more position sensors 135, andan inertial measurement unit (IMU) 140. Some embodiments of The HMD 105have different components than those described in conjunction withFIG. 1. Additionally, the functionality provided by various componentsdescribed in conjunction with FIG. 1 may be differently distributedamong the components of the HMD 105 in other embodiments.

The DCA 120 captures data describing depth information of an areasurrounding the HMD 105. Some embodiments of the DCA 120 include one ormore imaging devices (e.g., a camera, a video camera) and anillumination source configured to emit a series of periodic illuminationpatterns, with each periodic illumination pattern phase shifted by adifferent amount. As another example, the illumination source emits aseries of sinusoids that each have a specific spatial phase shift. Theperiodicity of the illumination pattern is a spatial periodicityobserved on an object illuminated by the illumination pattern, and thephase shifts are lateral spatial phase shifts along the direction ofperiodicity. In various embodiments, the periodicity of the illuminationpattern is in a direction that is parallel to a displacement between theillumination source and a center of the imaging device of the DCA 120

For example, the illumination source emits a series of sinusoids thateach have a different spatial phase shift into an environmentsurrounding the HMD 105. In other examples, the illumination sourceemits a sinusoidal pattern multiplied by a low frequency envelope, suchas a Gaussian, which changes relative signal intensity over the field ofview of the imaging device. This change in relative signal intensityover the imaging device's field of view changes temporal noisecharacteristics without affecting the depth determination, which isfurther described below in conjunction with FIGS. 4 and 5 provided thehigher frequency signal is a sinusoid. The imaging device of the DCA 120includes a sensor comprising multiple pixels that determine a phaseshift of a periodic illumination pattern included in multiple imagescaptured by the imaging device based on relative intensities included inthe multiple captured images. As the phase shift is a function of depth,the DCA 120 determines a depth of a location within the local area fromwhich images of the periodic illumination from the determined phaseshift, as further described below in conjunction with FIGS. 4 and 5. Invarious embodiments, each pixel of the sensor of the imaging devicedetermines a depth of a location within the local area from which apixel captured intensities of the periodic illumination pattern based ona phase shift determined for the periodic illumination pattern capturedby the pixel.

The imaging device captures and records particular ranges of wavelengthsof light (i.e., “bands” of light). Example bands of light captured by animaging device include: a visible band (˜380 nm to 750 nm), an infrared(IR) band (˜750 nm to 2,200 nm), an ultraviolet band (100 nm to 380 nm),another portion of the electromagnetic spectrum, or some combinationthereof. In some embodiments, an imaging device captures imagesincluding light in the visible band and in the infrared band. In variousembodiments, the imaging device captures light in bands of wavelengthsbased on a wavelength of light emitted by the illumination source. Forexample, the illumination source emits light having a wavelengths within10 nm of a wavelength emitted by the illumination source; so if theillumination source emits light having wavelengths of 850 nm, 940 nm,and 1550 nm, the imaging device captures light having wavelengthsbetween 840 nm and 860 nm, between 930 nm and 950 nm, and between 1540nm and 1560 nm. In another example the illumination source emits lighthaving wavelengths of 820 nm, 850 nm, and 940 nm, the imaging devicecaptures light having wavelengths between 810 nm and 830 nm, between 840nm and 860 nm, and between 930 nm and 950 nm. In other embodiments, anysuitable combination of wavelengths may be used by the illuminationsource to emit the periodic illumination pattern, so the imaging devicemay capture wavelengths corresponding to different colors in variousembodiments.

The electronic display 125 displays 2D or 3D images to the user inaccordance with data received from the console 110. In variousembodiments, the electronic display 125 comprises a single electronicdisplay or multiple electronic displays (e.g., a display for each eye ofa user). Examples of the electronic display 125 include: a liquidcrystal display (LCD), an organic light emitting diode (OLED) display,an active-matrix organic light-emitting diode display (AMOLED), someother display, or some combination thereof.

The optics block 130 magnifies image light received from the electronicdisplay 125, corrects optical errors associated with the image light,and presents the corrected image light to a user of the HMD 105. Invarious embodiments, the optics block 130 includes one or more opticalelements. Example optical elements included in the optics block 130include: an aperture, a Fresnel lens, a convex lens, a concave lens, afilter, a reflecting surface, or any other suitable optical element thataffects image light. Moreover, the optics block 130 may includecombinations of different optical elements. In some embodiments, one ormore of the optical elements in the optics block 130 may have one ormore coatings, such as anti-reflective coatings.

Magnification and focusing of the image light by the optics block 130allows the electronic display 125 to be physically smaller, weigh lessand consume less power than larger displays. Additionally, magnificationmay increase the field of view of the content presented by theelectronic display 125. For example, the field of view of the displayedcontent is such that the displayed content is presented using almost all(e.g., approximately 110 degrees diagonal), and in some cases all, ofthe user's field of view. Additionally in some embodiments, the amountof magnification may be adjusted by adding or removing optical elements.

In some embodiments, the optics block 130 may be designed to correct oneor more types of optical error. Examples of optical error include barreldistortions, pincushion distortions, longitudinal chromatic aberrations,or transverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, comatic aberrations or errors dueto the lens field curvature, astigmatisms, or any other type of opticalerror. In some embodiments, content provided to the electronic display125 for display is pre-distorted, and the optics block 130 corrects thedistortion when it receives image light from the electronic display 125generated based on the content.

The IMU 140 is an electronic device that generates data indicating aposition of the HMD 105 based on measurement signals received from oneor more of the position sensors 135 and from depth information receivedfrom the DCA 120. A position sensor 135 generates one or moremeasurement signals in response to motion of the HMD 105. Examples ofposition sensors 135 include: one or more accelerometers, one or moregyroscopes, one or more magnetometers, another suitable type of sensorthat detects motion, a type of sensor used for error correction of theIMU 140, or some combination thereof. The position sensors 135 may belocated external to the IMU 140, internal to the IMU 140, or somecombination thereof.

Based on the one or more measurement signals from one or more positionsensors 135, the IMU 140 generates data indicating an estimated currentposition of the HMD 105 relative to an initial position of the HMD 105.For example, the position sensors 135 include multiple accelerometers tomeasure translational motion (forward/back, up/down, left/right) andmultiple gyroscopes to measure rotational motion (e.g., pitch, yaw,roll). In some embodiments, the IMU 140 rapidly samples the measurementsignals and calculates the estimated current position of the HMD 105from the sampled data. For example, the IMU 140 integrates themeasurement signals received from the accelerometers over time toestimate a velocity vector and integrates the velocity vector over timeto determine an estimated current position of a reference point on theHMD 105. Alternatively, the IMU 140 provides the sampled measurementsignals to the console 110, which interprets the data to reduce error.The reference point is a point that may be used to describe the positionof the HMD 105. The reference point may generally be defined as a pointin space or a position related to the HMD's 105 orientation andposition.

The IMU 140 receives one or more parameters from the console 110. Asfurther discussed below, the one or more parameters are used to maintaintracking of the HMD 105. Based on a received parameter, the IMU 140 mayadjust one or more IMU parameters (e.g., sample rate). In someembodiments, certain parameters cause the IMU 140 to update an initialposition of the reference point so it corresponds to a next position ofthe reference point. Updating the initial position of the referencepoint as the next calibrated position of the reference point helpsreduce accumulated error associated with the current position estimatedthe IMU 140. The accumulated error, also referred to as drift error,causes the estimated position of the reference point to “drift” awayfrom the actual position of the reference point over time. In someembodiments of the HMD 105, the IMU 140 may be a dedicated hardwarecomponent. In other embodiments, the IMU 140 may be a software componentimplemented in one or more processors.

The I/O interface 115 is a device that allows a user to send actionrequests and receive responses from the console 110. An action requestis a request to perform a particular action. For example, an actionrequest may be an instruction to start or end capture of image or videodata or an instruction to perform a particular action within anapplication. The I/O interface 115 may include one or more inputdevices. Example input devices include: a keyboard, a mouse, a gamecontroller, or any other suitable device for receiving action requestsand communicating the action requests to the console 110. An actionrequest received by the I/O interface 115 is communicated to the console110, which performs an action corresponding to the action request. Insome embodiments, the I/O interface 115 includes an IMU 140, as furtherdescribed above, that captures calibration data indicating an estimatedposition of the I/O interface 115 relative to an initial position of theI/O interface 115. In some embodiments, the I/O interface 115 mayprovide haptic feedback to the user in accordance with instructionsreceived from the console 110. For example, haptic feedback is providedwhen an action request is received, or the console 110 communicatesinstructions to the I/O interface 115 causing the I/O interface 115 togenerate haptic feedback when the console 110 performs an action.

The console 110 provides content to the HMD 105 for processing inaccordance with information received from one or more of: the DCA 120,the HMD 105, and the VR I/O interface 115. In the example shown in FIG.1, the console 110 includes an application store 150, a tracking module155 and a content engine 145. Some embodiments of the console 110 havedifferent modules or components than those described in conjunction withFIG. 1. Similarly, the functions further described below may bedistributed among components of the console 110 in a different mannerthan described in conjunction with FIG. 1.

The application store 150 stores one or more applications for executionby the console 110. An application is a group of instructions, that whenexecuted by a processor, generates content for presentation to the user.Content generated by an application may be in response to inputsreceived from the user via movement of the HMD 105 or the I/O interface115. Examples of applications include: gaming applications, conferencingapplications, video playback applications, or other suitableapplications.

The tracking module 155 calibrates the system environment 100 using oneor more calibration parameters and may adjust one or more calibrationparameters to reduce error in determination of the position of the HMD105 or of the I/O interface 115. For example, the tracking module 155communicates a calibration parameter to the DCA 120 to adjust the focusof the DCA 120 to more accurately determine depths of locations withinthe local area surrounding the HMD 105 from captured intensities.Calibration performed by the tracking module 155 also accounts forinformation received from the IMU 140 in the HMD 105 and/or an IMU 140included in the I/O interface 115. Additionally, if tracking of the HMD105 is lost (e.g., the DCA 120 loses line of sight of at least athreshold number of SL elements), the tracking module 140 mayre-calibrate some or all of the system environment 100.

The tracking module 155 tracks movements of the HMD 105 or of the I/Ointerface 115 using information from the DCA 120, the one or moreposition sensors 135, the IMU 140 or some combination thereof. Forexample, the tracking module 155 determines a position of a referencepoint of the HMD 105 in a mapping of a local area based on informationfrom the HMD 105. The tracking module 155 may also determine positionsof the reference point of the HMD 105 or a reference point of the I/Ointerface 115 using data indicating a position of the HMD 105 from theIMU 140 or using data indicating a position of the I/O interface 115from an IMU 140 included in the I/O interface 115, respectively.Additionally, in some embodiments, the tracking module 155 may useportions of data indicating a position of the HMD 105 from the IMU 140as well as representations of the local area from the DCA 120 to predicta future location of the HMD 105. The tracking module 155 provides theestimated or predicted future position of the HMD 105 or the I/Ointerface 115 to the content engine 145.

The content engine 145 generates a 3D mapping of the area surroundingthe HMD 105 (i.e., the “local area”) based on information received fromthe DCA 120 included in the HMD 105. In some embodiments, the contentengine 145 determines depth information for the 3D mapping of the localarea based on depths determined by each pixel of the sensor in theimaging device from a phase shift determined from relative intensitiescaptured by a pixel of the sensor in multiple images. In variousembodiments, the content engine 145 uses different types of informationdetermined by the DCA 120 or a combination of types of informationdetermined by the DCA 120 to generate the 3D mapping of the local area.

The content engine 145 also executes applications within the systemenvironment 100 and receives position information, accelerationinformation, velocity information, predicted future positions, or somecombination thereof, of the HMD 105 from the tracking module 155. Basedon the received information, the content engine 145 determines contentto provide to the HMD 105 for presentation to the user. For example, ifthe received information indicates that the user has looked to the left,the content engine 145 generates content for the HMD 105 that mirrorsthe user's movement in a virtual environment or in an environmentaugmenting the local area with additional content. Additionally, thecontent engine 145 performs an action within an application executing onthe console 110 in response to an action request received from the I/Ointerface 115 and provides feedback to the user that the action wasperformed. The provided feedback may be visual or audible feedback viathe HMD 105 or haptic feedback via the I/O interface 115.

Head Mounted Display

FIG. 2 is a wire diagram of one embodiment of a head mounted display(HMD) 200. The HMD 200 is an embodiment of the HMD 105, and includes afront rigid body 205, a band 210, a reference point 215, a left side220A, a top side 220B, a right side 220C, a bottom side 220D, and afront side 220E. The HMD 200 shown in FIG. 2 also includes an embodimentof a depth camera assembly (DCA) 120 including an imaging device 225 andan illumination source 230, which are further described below inconjunction with FIGS. 3 and 4. The front rigid body 205 includes one ormore electronic display elements of the electronic display 125 (notshown), the IMU 130, the one or more position sensors 135, and thereference point 215.

In the embodiment shown by FIG. 2, the HMD 200 includes a DCA 120comprising an illumination source 225, such as a camera, and anillumination source 230 configured to emit a series of periodicillumination patterns, with each periodic illumination pattern phaseshifted by a different amount into a local area surrounding the HMD 200.In various embodiments, the illumination source 230 emits a sinusoidalpattern, a near sinusoidal pattern, or any other periodic pattern (e.g.,a square wave). For example, the illumination source 230 emits a seriesof sinusoids that each have a different phase shift into an environmentsurrounding the HMD 200. In various embodiments, the illumination source230 includes an acousto-optic modulator configured to generate twoGaussian beams of light that interfere with each other in the local areaso a sinusoidal interference pattern is generated. However, in otherembodiments the illumination source 230 includes one or more of anacousto-optic device, an electro-optic device, physical optics, opticalinterference, a diffractive optical device, or any other suitablecomponents configured to generate the periodic illumination pattern. Inother examples, the illumination source 230 comprises multiple photonsources each emitting different wavelengths that are positioned within athreshold distance of each other and positioned to emit photons havingdifferent wavelengths that originate from a specific physical locationwhere the multiple photon sources are positioned. In some embodiments,the illumination source 230 includes additional optical elements thatmodify the generated sinusoidal interference pattern to be within anintensity envelope (e.g., within a Gaussian intensity pattern);alternatively, the HMD 200 includes the additional optical elements andthe Gaussian beams of light generated by the illumination source 230 aredirected through the additional optical elements before being emittedinto the environment surrounding the HMD 200. The imaging device 225captures images of the local area, which are used to calculate depthsrelative to the HMD 200 of various locations within the local area, asfurther described below in conjunction with FIGS. 3-5.

FIG. 3 is a cross section of the front rigid body 205 of the HMD 200depicted in FIG. 2. As shown in FIG. 3, the front rigid body 205includes an imaging device 225 and an illumination source 230. The frontrigid body 205 also has an optical axis corresponding to a path alongwhich light propagates through the front rigid body 205. In someembodiments, the imaging device 225 is positioned along the optical axisand captures images of a local area 305, which is a portion of anenvironment surrounding the front rigid body 205 within a field of viewof the imaging device 225. Additionally, the front rigid body 205includes the electronic display 125 and the optics block 130, which arefurther described above in conjunction with FIG. 1. The front rigid body205 also includes an exit pupil 335 where the user's eye 340 is located.For purposes of illustration, FIG. 3 shows a cross section of the frontrigid body 205 in accordance with a single eye 340. The local area 305reflects incident ambient light as well as light projected by theillumination source 230, which is subsequently captured by the imagingdevice 225.

As described above in conjunction with FIG. 1, the electronic display125 emits light forming an image toward the optics block 130, whichalters the light received from the electronic display 125. The opticsblock 130 directs the altered image light to the exit pupil 335, whichis a location of the front rigid body 205 where a user's eye 340 ispositioned. FIG. 3 shows a cross section of the front rigid body 205 fora single eye 340 of the user, with another electronic display 125 andoptics block 130, separate from those shown in FIG. 3, included in thefront rigid body 205 to present content, such as an augmentedrepresentation of the local area 305 or virtual content, to another eyeof the user.

In the example shown by FIG. 3, the front rigid body 205 includes a lens302 that allowing the user's eye 340 to view the local area 305 whilethe user is wearing the HMD 200. For example, the lens 302 is atransparent material or a translucent material through which light fromthe local area 305 may be transmitted. The lens 302 allows the user toview the local area 305 as well as content generated by the electronicdisplay 125. Hence, the example shown by FIG. 3 depicts a HMD 200providing a user with augmented reality content. In other embodiments,the HMD 200 does not include the lens 302, which shields the user fromviewing the local area 305 while viewing content generated by theelectronic display 125.

As further described above in conjunction with FIG. 2, the illuminationsource 230 of the depth camera assembly (DCA) emits a series of periodicillumination patterns, with each periodic illumination pattern phaseshifted by a different amount into the local area 305, and the imagingdevice 225 captures images of the periodic illumination patternsprojected onto the local area 305 using a sensor comprising multiplepixels. Each pixel captures intensity of light emitted by theillumination source 230 from the local area 305 in various images andcommunicates the captured intensity to a controller or to the console110, which determines a phase shift for each image, as further describedbelow in conjunction with FIGS. 4-6B, and determines a depth of alocation within the local area onto which the light emitted by theillumination source 230 captured by the imaging device 225 was captured,also further described below in conjunction with FIGS. 4-6B.

Depth Camera Assembly

FIG. 4 example of light emitted into a local area and captured by adepth camera assembly included in a head mounted display (HMD) 105. FIG.4 shows an imaging device 225 and an illumination source 230 of a depthcamera assembly (DCA) 120 included in the HMD. As shown in FIG. 4,imaging device 225 and the illumination source 230 are separated by aspecific distance D (also referred to as a “baseline”), which isspecified when the DCA 120 is assembled. The distance D between theimaging device 223 and the illumination source 230 is stored in astorage device coupled to the imaging device 225, coupled to acontroller included in the DCA 120, or coupled to the console 110 invarious embodiments.

In the example of FIG. 4, the illumination source 230 emits a smoothcontinuous intensity pattern of light 405 onto a flat target 410 withina local area surrounding the HMD 105 and within a field of view of theimaging device 225. The continuous intensity pattern of light 405 has aperiod T known to the DCA 120. However, in other embodiments, theillumination source 230 emits any suitable intensity pattern having aperiod T known to the DCA 120. Additionally, FIG. 4 identifies an angleθ, that is one half of the period T of the continuous intensity patternof light 405. As the continuous intensity pattern of light 405 scaleslaterally with the depth from the DCA 120, θ_(i) defines a depthindependent periodicity of the illumination. Similarly, FIG. 4 depictsan angle θ_(c) and a line perpendicular to a plane including the imagingdevice 225 and a location on the target 410 from which a particularpixel of a sensor included in the imaging device 225 capturesintensities of the continuous intensity pattern of light 405 indifferent images; hence, θ_(c) specifies an angle between the lineperpendicular to the plane including the imaging device 225 and thelocation on the target 410 from which the specific pixel capturesintensities of the continuous intensity pattern of light 405 emitted bythe illumination source 230.

Each pixel of the sensor of the imaging device 225 provides an intensityof light from the continuous intensity pattern of light 405 captured inmultiple images to a controller or to the console 110, which determinesa phase shift, ϕ, of the continuous intensity pattern of light 405captured by each pixel of the sensor. Each image captured by the imagingdevice 225 is a digital sampling of the continuous intensity pattern oflight 405, so the set of images captured by the sensor represent aFourier transform of the continuous intensity pattern of light 405, andthe Fourier components, a₁ and b₁, of the fundamental harmonic of thecontinuous intensity pattern 405 are directly related to the phase shiftfor a pixel of the sensor. For images captured by a pixel of the sensor,the Fourier components a₁ and b₁ are determined using the followingequations:

$\begin{matrix}{a_{1} = {\sum\limits_{n = 1}^{N}{S_{n}{\cos\left( \theta_{n} \right)}{\Delta\theta}}}} & (1) \\{b_{1} = {\sum\limits_{n = 1}^{N}{S_{n}{\sin\left( \theta_{n} \right)}{\Delta\theta}}}} & (2)\end{matrix}$

In the preceding, S_(n) denotes an intensity of the pixel of the sensorin a particular image, n, captured by the sensor, and the set θn ofrepresents the phase shifts introduced into the continuous intensitypattern of light 405. For example, if three phase shifts are used, theset of θn includes 0 degrees, 120 degrees, and 240 degrees. As anotherexample, if four phase shifts are used the set of θn includes 0 degrees,90 degrees, 180 degrees, and 270 degrees. In some embodiments, the setof θ_(n) is determines so 0 degrees and 360 degrees are uniformlysampled by the captured images, but the set of θ_(n) may include anyvalues in different implementations.

From the Fourier components a₁ and b₁ determined as described above, thecontroller or the console determines the phase shift (I) of thecontinuous intensity pattern of light 405 captured by a pixel of thesensor as follows:

$\begin{matrix}{{\varnothing(R)} = {{\tan^{- 1}\left( \frac{a_{1}}{b_{1}} \right)} - \theta_{1}}} & (3) \\{R = \sqrt{a_{1}^{2} + b_{1}^{2}}} & (4)\end{matrix}$

In the preceding, ϕ is the phase shift of the first harmonic of thecontinuous intensity pattern of light 405, R is the magnitude of thefirst harmonic of the continuous intensity pattern of light 405, and θ₁is a calibration offset. For each spatial frequency of the continuousintensity pattern of light 405, the DCA 120 determines phase shiftsusing the intensity of the pixel of the sensor in at least three images.

The phase shift of the first harmonic of the continuous intensitypattern 405 determined through equation (3) above is used by acontroller 430 coupled to the imaging device 225 and to the illuminationsource 230. In various embodiments the controller 430 is a processorthat may be included in the imaging device 225, in the illuminationsource 230, or in the console 110 to determine the depth of the locationof the target 410 from which the pixel of the sensor capturesintensities of the continuous intensity pattern of light 405 as follows:

$\begin{matrix}{z = \frac{D}{{\frac{\tan\left( \theta_{i} \right)}{\pi}\left( {\varnothing_{ij} - \varnothing_{{ij},{cal}}} \right)} - {\tan\left( \theta_{c} \right)}}} & (5)\end{matrix}$Where z is the depth of the location of the target 410 from which thepixel of the sensor captures intensities of the continuous intensitypattern of light 405; D is the distance between the illumination source230 and the imaging device 225; θ_(i) is one half of the period T of thecontinuous intensity pattern of light 405; and θ_(c) is an angle betweenand a line perpendicular to a plane including the imaging device 225 anda the location on the target 410 from which a particular pixel locatedat row i and column j of the sensor included in the imaging device 225captured intensities of the continuous intensity pattern of light 405.Additionally, ϕ_(ij) is the phase shift determined for the pixel at rowi and column j of the sensor, determined as further described above.Further, ϕ_(ij,cal) is a calibration offset for the pixel of the sensorat row i and column j of the sensor, which is determined as furtherdescribed below.

The DCA 120 determines phase shifts for each of at least a set of pixelsof the sensor of the imaging device 225, as described above. For each ofat least the set of pixels, the DCA 120 determines a depth from the DCA120 to a location within the local area surrounding the DCA 120 fromwhich a pixel of the set captured intensities of the continuousintensity pattern of light 405 emitted into the local area. This allowsdifferent pixels of the sensor of the imaging device 225 to determinedepths of locations within the local area from which different pixelscaptured intensities of the continuous intensity pattern of light 405.In some embodiments, each pixel of the sensor of the imaging device 225determines a depth from the DCA 120 to a location within the local areasurrounding the DCA 120 from which a pixel captured intensities of thecontinuous intensity pattern of light 405 in various images. The DCA 120may generate a depth map identifying depths from the DCA 120 todifferent locations within the local area from which different pixelscaptured intensities of the continuous intensity pattern of light 405.For example, the generated depth map identifies depths from the DCA 120to different locations within the local area based on intensitiescaptured by each pixel of the sensor, with a depth corresponding to apixel of the sensor that captured intensities used to determine thedepth.

However, because the phase shift is within a range of 0 and 2π radians,there may be ambiguities in resolving phase shifts that are integermultiples of 2π when determining the phase shift as described above. Toavoid these potential ambiguities, in some embodiments, the continuousintensity pattern of light 405 emitted by the illumination source 230 asa single, relatively lower, spatial frequency; however, use of arelatively lower spatial frequency may decrease precision of the depthdetermination by the DCA 120. Alternatively, the continuous intensitypattern of light 405 includes two or more spatial frequencies insequence. Using two or more spatial frequencies increases a range ofphases within which phase shifts may be unambiguously identified. Therange of phases is extended for a subset of pixels within the sensor ofthe imaging device 225 based on a maximum parallax expected duringoperation of the imaging device 225, which may be determined based on adifference between a maximum range and a minimum range of the imagingdevice 225. Hence, the range of phases is extended for the subset ofpixels of the sensor most likely to capture light from the continuousintensity pattern of light 405.

FIG. 5 shows an example of using two frequencies of a continuousintensity pattern of light emitted by a DCA 120 to identify a phaseshift for a pixel of the sensor. In the example of FIG. 5, phase shiftsidentified from frequency 505 repeat through the interval of 0 and 2πradians three times in a time interval, while phase shifts identifiedfrom frequency 510 repeat through the interval of 0 and 2π radians twicein the time interval, as shown in plot 520. Hence, emitting lightpatterns having frequency 505 and frequency 510 allows the DCA 120 toidentify a phase shift in the time interval over a larger interval thanbetween 0 and 2π (i.e., “unwraps” the phase shifts that may beunambiguously identified). FIG. 5 shows another example where, phaseshifts identified from frequency 505 repeat through the interval of 0and 2π radians five times in a time interval, while phase shiftsidentified from frequency 515 repeat through the interval of 0 and 2πradians twice in the time interval, as shown in plot 530. This similarlyallows the DCA 120 to identify a phase shift in the time interval over alarger interval than between 0 and 2π (i.e., “unwraps” the phase shiftsthat may be unambiguously identified). Additionally, FIG. 5 also showsan analogous three dimensional plot 540 of frequency 505, frequency 510,and frequency 515, which may further extend the range of phases overwhich phase shifts may be unambiguously identified. In otherembodiments, any number of frequencies of the continuous intensitypattern of light may be used to identify the phase shift for the pixelof the sensor using the process further described above.

Depth Camera Assembly Calibration

Referring again to FIG. 4, a pixel of the sensor of the imaging device225 captures intensity of the continuous intensity pattern of light 405at a position of D+x₀ relative to the illumination source 230, where x₀is a distance from a principal point of the imaging device 225 along anaxis separating the illumination source 230 and the sensor (e.g., alonga horizontal axis along which the illumination source 230 and the sensorare positioned). As further described above in conjunction with FIG. 4,the position of the pixel along the axis separating the illuminationsource 230 and the sensor is related to the phase shift, ϕ_(ij),determined for the pixel. Additionally, as further described above θ_(i)defines the spatial periodicity of the continuous intensity pattern oflight 405 in the local area and corresponds to half of the period T ofthe continuous intensity pattern of light. As the continuous intensitypattern of light 405 expands angularly as depth z from the DCA 120increases, the period T of the continuous intensity pattern of light 405corresponds to a specific depth z from the DCA 120, while theperiodicity defined by θ_(i) is independent of depth z from the DCA 120.The dependence of the period T of the continuous intensity pattern oflight 405, in combination with the distance D between the imaging device225 and the illumination source 230 allows the DCA 120 to determine thedepth z of an object onto which the continuous intensity pattern oflight 405 is emitted, as the lateral distance at which the pixelcaptures a phase, D+x₀, is equal to a product of the period T of thecontinuous intensity pattern of light 405 captured by the imaging device225 and a ratio of the phase shift, ϕ_(ij), determined for the pixel toa (i.e., D+x₀=T(ϕ_(ij)/2π). This relationship between thedepth-dependent period T, the distance from a principal point of theimaging device 225, and phase shift, ϕ_(ij), determined for the pixelequates a an estimate of lateral extent at the camera plane and theplane including the object onto which the continuous intensity patternof light 405 was emitted, which both measure a distance from a center ofthe continuous intensity pattern of light 405 to a central ray of thepixel.

The continuous intensity pattern of light 405 may be calibrated ordetermined using any suitable method, and scales with depth from the DCA120. Accordingly, the period T of the continuous intensity pattern oflight 405 at the depth z from the DCA 120 is equal to double a productof the depth z form the DCA 120 and a tangent of the angle, θ_(i), whichdefines half of the period T of the continuous intensity pattern oflight (i.e., T=(2)(z)(tan(θ_(i)))). Similarly, the location of the pixelrelative to the illumination source 230 along an axis separating theillumination source 230 and the sensor, x₀, is a product of the depthfrom the DCA 120, z. and a tangent of the angle, θ_(c), between the lineperpendicular to the plane including the imaging device 225 and thelocation on the target 410 from which the specific pixel capturesintensities of the continuous intensity pattern of light 405 emitted bythe illumination source 230 (i.e., x₀=z(tan(θ_(c)))). Accordingly,

$\begin{matrix}{{D + {z\left( {\tan\theta_{c}} \right)}} = {2{z\left( {\tan\theta_{i}} \right)}\left( \frac{\varnothing_{ij}}{2\pi} \right)}} & (6)\end{matrix}$

Solving equation 6 above for depth, z:

$\begin{matrix}{z = \frac{D}{{\frac{\tan\;\theta_{i}}{\pi}\varnothing_{ij}} - {\tan\theta_{c}}}} & (7)\end{matrix}$

However, equation 7 above is based on the phase shift, ϕ_(ij), when thelocation. x₀, of the pixel relative to the illumination source 230 alongequals the inverse of the specific distance D separating the imagingdevice 225 and the illumination source 230 is zero (i.e.,ϕ_(ij)(x₀=D)=0). To satisfy this condition, a calibration offset,ϕ_(ij,cal), is determined for each pixel via a calibration process wherethe sensor of the imaging device 225 captures intensities from thecontinuous illumination pattern of light 405 emitted onto a target at anaccurately predetermined depth, z_(cal). In various embodiments, thetarget is a Lambertian surface or other surface that reflects at least athreshold amount of light incident on the target. Accounting for thecalibration offset modifies equation (7) above into equation (5),

${z = \frac{D}{{\frac{\tan\left( \theta_{i} \right)}{\pi}\left( {\varnothing_{ij} - \varnothing_{{ij},{cal}}} \right)} - {\tan\left( \theta_{c} \right)}}},$which was previously described above in conjunction with FIG. 4. Withthe predetermined depth, z_(cal), the calibration offset for each pixelis determined as:

$\begin{matrix}{\varnothing_{{ij},{cal}} = {\varnothing_{ij} - {\frac{\pi}{\tan\theta_{i}}\left\lbrack {\frac{D}{z_{cal}} + {\tan\theta_{c}}} \right\rbrack}}} & (8)\end{matrix}$

The calibration offset is determined for each pixel of the sensor andfor each frequency of the continuous intensity pattern of light 405based on the predetermined depth z_(cal) and is stored in the DCA 120for use during operation. A calibration offset for each pixel of thesensor is determined for each period of continuous intensity pattern oflight 405 emitted by the illumination source 230 and stored during thecalibration process. For example, the DCA 120 stores a calibrationoffset for a pixel of the sensor in association with a location (e.g., arow and a column) of the pixel within the sensor and in association witha frequency of the continuous intensity pattern of light 405. In variousembodiments, the DCA 120 stores a parameterized function for determiningthe calibration offset of different pixels of the sensor based onlocation within the sensor and frequency of the continuous intensitypattern of light 405 instead of storing calibration offsets determinedfor individual pixels of the sensor. The DCA 120 stores a parameterizedfunction corresponding to each period T of continuous intensity patternsof light 405 emitted by the illumination source 230 in variousembodiments. In some embodiments, the parameterized function determiningthe calibration offset of different pixels is a linear function.

In embodiments where the illumination source 230 includes anacousto-optic modulator configured to generate two Gaussian beams oflight that interfere with each other in the local area so a sinusoidalinterference pattern is generated as the continuous intensity pattern oflight 405 emitted into the local area, the period T of the continuousintensity pattern of light 405 is determined as:

$\begin{matrix}{\frac{T}{2} = \frac{z}{\sqrt{\left( \frac{2a}{\lambda} \right)^{2} - 1}}} & (9)\end{matrix}$

In equation 9, λ is a wavelength of the illumination source 230 and a isthe separation of the Gaussian beams generated by the acousto-opticmodulator to generate the continuous intensity pattern of light 405emitted into the local area surrounding the DCA 120. The determinedperiod T may then be used to determine the calibration offset forvarious pixels of the detector, as further described above.

Imaging Device Sensor

FIG. 6 shows an example sensor 600 included in an imaging device 225 ofa depth camera assembly (DCA) 120. In the example of FIG. 6, thecontinuous intensity pattern of light includes a plurality ofwavelengths, so the illumination source 230 emits the plurality ofwavelengths simultaneously or sequentially. In various embodiments, theillumination source 230 emits three wavelengths comprising thecontinuous intensity pattern of light sequentially or simultaneously;however, in other embodiments, the continuous intensity pattern of lightincludes any suitable number of wavelengths, which are sequentially orsimultaneously by the illumination source 230. Accordingly, the sensor600 of the imaging device 225 includes optical filters corresponding towavelengths of light emitted by the illumination source 230 that arespatially distributed across the sensor 600 to form regions 605A, 605B,605C (also referred to individually and collectively using referencenumber 605) that each comprise at least one of the optical filters. Eachregion 605 includes adjacent pixels 610A, 610B, 610C, 610D of the sensor600. In the example of FIG. 6, three regions 605A, 605B, 605C areidentified, corresponding to three narrow optical filters. If theillumination source 230 emits a smoothly-varying continuous intensitypattern of light that is a sinusoid or approximates a sinusoid, thesensor includes three narrow optical filters as shown in FIG. 6.However, in embodiments where the illumination source 230 emits othercontinuous intensity patterns of light (e.g., a square wave), the sensor600 includes an increased number of narrow optical filters to reduceharmonic errors in disparity estimation; accordingly, the sensor 600 mayinclude any suitable number of narrow optical filters corresponding toregions 605.

If the illumination source 230 emits multiple intensity patterns havinga common spatial frequency, different phase shifts, and non-overlappingwavelengths simultaneously, different regions 605A, 605B, 605Csimultaneously capture the intensity profile of the emitted wavelengths,as different regions 605A, 605B, 605C correspond to a bandpass filtersconfigured to capture different wavelengths. For example, if theillumination source 230 emits three non-overlapping wavelengths, region605A corresponds to an optical filter configured to pass wavelengthsincluding a first wavelength emitted by the illumination source 230,region 605B corresponds to an alternative optical filter configured topass wavelengths including a second wavelength emitted by theillumination source 230, and region 605C corresponds to an additionaloptical filter configured to pass wavelengths including a thirdwavelength emitted by the illumination source 230. Accordingly, pixels610 a region 605A capture intensity of a wavelength emitted by theillumination source 230 and passed by the optical filter correspondingto the region 605A. In the preceding example, pixels 610A, 610B, 610C,610D included in region 605A capture intensity of the first wavelength,while pixels 610A, 610B, 610C, 610D included in region 605C captureintensity of the second wavelength, and pixels 610A, 610B, 610C, 610Dincluded in region 605A capture intensity of the third wavelength. WhileFIG. 6 shows an example with three regions 605A, 605B, 605C forcapturing three different wavelengths emitted by the illumination source230, in other embodiments, the illumination source 230 emits anysuitable number of different wavelengths and the sensor 600 includes anumber of regions 605 corresponding to the number of differentwavelengths emitted by the illumination source 230.

Alternatively, the illumination source 230 is configured to sequentiallyemit different wavelengths at different times. In some embodiments, thesensor 600 and the illumination source 230 are gated, so the sensor 600captures intensity information while the illumination source 230 isemitting light and does not capture intensity information while theillumination source 230 is not emitting light. In other embodiments,different regions 605A, 605B, 605C of the sensor 600 are gated to theillumination source 230, so a region 605 corresponding to a particularwavelength captures intensity information when the illumination source230 emits the particular wavelength, while other regions 605corresponding to optical filters passing through other wavelengths donot capture intensity information while the illumination source 230emits the particular wavelength. For example, the illumination source230 and the imaging device 225 are communicatively coupled and theillumination source 230 transmits timing information to the imagingdevice 225 specifying times when the illumination source 230 emitsillumination patterns having different wavelengths are emitted. Based onthe timing information, the imaging device 225 modifies regions 605 ofthe sensor 600 so one or more regions 605 of pixels 610 corresponding toa particular wavelength emitted by the illumination source 230 during atime interval capture intensity information, while other regions 605corresponding to different wavelengths do not capture intensityinformation. Hence, different regions 605 of the sensor 600 captureintensity information at different times based on when the illuminationsource 230 emits illumination patterns having different wavelengths.Having a region 605 capture intensity information when the illuminationsource 230 emits a specific wavelength and not capture intensityinformation when the illumination source 230 emits other wavelengthsreduces an amount of ambient photons collected in pixels 610 within theregion 605 when the illumination source 230 is not emitting the specificwavelength corresponding to the region 605.

In embodiments where the illumination source 230 includes anacousto-optic modulator to generate the continuous intensity patterns oflight, the illumination source 230 generates an analog illuminationpattern, allowing the illumination pattern to be sinusoidal withouthigher-order harmonics. However, to generate different phase shifts fordifferent illumination patterns, an acoustic signal travels along acrystal of the acousto-optic modulator, and the phase shift of anillumination pattern shift as the illumination pattern is within thelocal area. To maintain a single phase shift, the illumination source230 pulses a laser to be incident on the crystal of the acousto-opticmodulator that is in phase with an acoustic signal travelling along thecrystal, with the laser pulsed until the imaging device 225 has at leasta threshold signal to noise ratio. To generate three different intensitypatterns each having a specific phase shift, the illumination source 230may pulse three lasers that each have different wavelengths to beincident on the crystal of the acousto-optic modulator at differenttimes as an acoustic wave propagates through the crystal of theacousto-optic modulator.

The language used in the specification has been principally selected forreadability and instructional purposes, and it may not have beenselected to delineate or circumscribe the inventive subject matter. Itis therefore intended that the scope of the patent rights be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thepatent rights.

What is claimed is:
 1. A system comprising: an illumination sourceconfigured to emit light comprising a series of spatially periodicillumination patterns into a local area surrounding the illuminationsource, each periodic illumination pattern having a common spatialfrequency, a different phase shift, and a different wavelength; a sensorseparated from the illumination source by a specific distance, thesensor comprising: a plurality of pixels, each pixel configured tocapture intensity information based on light from the local area; a setof optical filters each positioned to correspond to a region includingadjacent pixels of the plurality of pixels, an optical filter configuredto pass light having a specific wavelength emitted by the illuminationsource to the pixels included in the region corresponding to the opticalfilter, where different optical filters pass different wavelengths oflight emitted by the illumination source to corresponding regions; afirst region configured to capture intensity information from lighthaving a first wavelength emitted by the illumination source while theillumination source emits a periodic illumination pattern having thefirst wavelength and not to capture intensity information while theillumination source emits one or more periodic illumination patternshaving wavelengths other than the first wavelength; and a second regionconfigured to capture intensity information from light having a secondwavelength emitted by the illumination source while the illuminationsource emits an illumination pattern having the second wavelength andnot to capture intensity information while the illumination source emitsone or more illumination patterns having wavelengths other than thesecond wavelength.
 2. The system of claim 1, wherein the illuminationsource is configured to emit three spatially periodic illuminationpatterns.
 3. The system of claim 2, wherein the sensor includes a firstoptical filter configured to pass light having the first wavelengthemitted by the illumination source to the first region of the sensor. 4.The system of claim 3, wherein the sensor further includes a secondoptical filter configured to pass light having the second wavelengthemitted by the illumination source to the second region of the sensor,the second region distinct from the first region.
 5. The system of claim3, wherein the sensor further includes a third optical filter configuredto pass light having a third wavelength emitted by the illuminationsource to a third region of the sensor, the third region distinct fromthe first region and from the second region.
 6. The system of claim 1,wherein the illumination source is configured to simultaneously emit aplurality of periodic illumination patterns each having differentwavelengths.
 7. A system comprising: an illumination source configuredto emit light comprising a series of spatially periodic illuminationpatterns into a local area surrounding the illumination source, eachperiodic illumination pattern having a common spatial frequency, adifferent phase shift, and a different wavelength; a sensor separatedfrom the illumination source by a specific distance and coupled to theillumination source, the sensor comprising: a plurality regions ofpixels, each region of pixels configured to capture intensityinformation based on light having a specific wavelength from the localarea, different regions capturing intensity information based ondifferent wavelengths of pixels, wherein a region capturing intensityinformation from a particular wavelength captures intensity informationduring a time interval when the illumination source emits a periodicillumination pattern having the particular wavelength and regionscapturing intensity information from other wavelengths do not captureintensity information during the time interval, a first regionconfigured to capture intensity information from light having a firstwavelength emitted by the illumination source while the illuminationsource emits a periodic illumination pattern having the first wavelengthand not to capture intensity information while the illumination sourceemits one or more periodic illumination patterns having wavelengthsother than the first wavelength; and a second region configured tocapture intensity information from light having a second wavelengthemitted by the illumination source while the illumination source emitsan illumination pattern having the second wavelength and not to captureintensity information while the illumination source emits one or moreillumination patterns having wavelengths other than the secondwavelength.
 8. The system of claim 7, wherein the illumination isconfigured to emit spatially periodic illumination patterns havingdifferent wavelengths at different time intervals.
 9. The system ofclaim 7, wherein the illumination source includes an acousto-opticmodulator.
 10. The system of claim 7, wherein a plurality of regionsconfigured to capture intensity information from light having aplurality of wavelengths are configured to capture intensity informationwhile the illumination source emits at least one periodic illuminationpattern and not to capture intensity information while the illuminationsource does not emit at least one periodic illumination pattern.
 11. Anapparatus comprising: a plurality of pixels, each pixel configured tocapture intensity information based on light from the local area; a setof optical filters each positioned to correspond to a region includingadjacent pixels of the plurality of pixels, an optical filter configuredto pass light having a specific wavelength emitted by the illuminationsource to the pixels included in the region corresponding to thebandpass filter, where different optical filters pass differentwavelengths of light emitted by the illumination source to correspondingregions; a first region configured to capture intensity information fromlight having a first wavelength emitted by an illumination source intothe local area while the illumination source emits a spatially periodicillumination pattern having the first wavelength and not to captureintensity information while the illumination source emits one or moreperiodic illumination patterns having wavelengths other than the firstwavelength; and a second region configured to capture intensityinformation from light having a second wavelength emitted by theillumination source into the local area while the illumination sourceemits a periodic illumination pattern having the second wavelength andnot to capture intensity information while the illumination source emitsone or more periodic illumination patterns having wavelengths other thanthe second wavelength.
 12. The apparatus of claim 11, wherein the set ofoptical filters includes a first bandpass filter configured to passlight having the first wavelength emitted to the first region of thesensor.
 13. The apparatus of claim 12, wherein the set of opticalfilters includes a second bandpass filter configured to pass lighthaving the second wavelength emitted by the illumination source to thesecond region of the sensor, the second region distinct from the firstregion.
 14. The apparatus of claim 13, wherein the set of opticalfilters further includes a third bandpass filter configured to passlight having a third wavelength emitted by the illumination source to athird region of the sensor, the third region distinct from the firstregion and from the second region.
 15. The apparatus of claim 11,wherein multiple regions corresponding to bandpass filters configured topass light having different wavelengths capture light during a timeinterval.
 16. An apparatus comprising: a plurality regions of pixels,each region of pixels configured to capture intensity information basedon light having a specific wavelength from the local area, differentregions capturing intensity information based on different wavelengthsof pixels, the plurality of regions communicatively coupled to anillumination source configured to emit illumination patterns havingdifferent wavelengths at different times, wherein a region capturingintensity information from a particular wavelength captures intensityinformation during a time interval when the illumination source emits aperiodic illumination pattern having the particular wavelength andregions capturing intensity information from other wavelengths do notcapture intensity information during the time interval, and theplurality of regions of pixels including: a first region configured tocapture intensity information from light having a first wavelengthemitted by the illumination source while the illumination source emits aspatially periodic illumination pattern having the first wavelength andnot to capture intensity information while the illumination source emitsone or more periodic illumination patterns having wavelengths other thanthe first wavelength; and a second region configured to captureintensity information from light having a second wavelength emitted bythe illumination source while the illumination source emits a periodicillumination pattern having the second wavelength and not to captureintensity information while the illumination source emits one or moreperiodic illumination patterns having wavelengths other than the secondwavelength.
 17. The apparatus of claim 16, wherein regions configured tocapture intensity information from light having a plurality of differentwavelengths are configured to capture intensity information while theillumination source emits at least one periodic illumination pattern andnot to capture intensity information while the illumination source doesnot emit at least one periodic illumination pattern.